Free page hinting is a mechanism which allows the guest driver to report
ranges of pages to the host device. A "hinting" run is triggered by the
device by issuing a new command id in the config space, after the update
to the id the device will hint ranges to the host which are unused. Once
the driver has exhausted all free ranges it notifies the device the run
has completed. The device can then issue another command allowing the
guest to reclaim these pages.

Adding support for hinting the firecracker balloon device, we offer
three points to manage the device; first to start a run, second to
monitor the status and a final to issue the command to allow the guest
to reclaim pages.

To note, there is a potential condition in the linux driver which would
allow a range to be reclaimed in an oom scenario before we remove the
range.

Signed-off-by: Jack Thomson <[email protected]>

Adding API endpoints to manage free page hinting . With
three different endpoint: Start - To begin a new run for free page
hinting, Status - To track the state of the hinting run, Stop - To stop
the hinting run and allow the guest to reclaim the pages reported.

Signed-off-by: Jack Thomson <[email protected]>

Add metrics to track free page hinting and reporting. For both devices
track the number of ranges reported, the number of errors encountered
while freeing and the total amount of memory freed.

Signed-off-by: Jack Thomson <[email protected]>

Adding new resources to the http api to enable testing of the hinting
functionality.

Signed-off-by: Jack Thomson <[email protected]>

Add option for fast_page_fault_helper to run in a oneshot mode, that
doesn't require the signal to be triggered before measuring the fault
latency.

This makes it possible to test fault latency on non-snapshotted vms as
only the first fault is measured.

Signed-off-by: Jack Thomson <[email protected]>

Add integration tests for free page hinting and reporting, both
functional and performance tests.

New functional tests to ensure that hinting and reporting are reducing
the RSS as expected in the guest. Updated reduce RSS test to touch
memory to reduce the chance of flakiness.

New performance tests for the balloon device. First being a test to
track the CPU overhead of hinting and reporting. Second being a test to
measure the faulting latency while reporting is running in the guest.

Signed-off-by: Jack Thomson <[email protected]>

Add integration tests for free page hinting and reporting. Asserting the
features are enabled correctly. Testing the config space updates
triggered by hinting are being set as expected.

Signed-off-by: Jack Thomson <[email protected]>

While the traditional balloon device would not be able to reclaim memory
when back by huge pages, it could still technically be used to to
restrict memory usage in the guest.

With the addition of hinting and reporting, they report ranges in bigger
sizes (4mb by default). Because of this, it is possible for the host
reclaim huge pages backing the guest.

Updates the performance tests for the balloon when back by huge pages,
added varients to the size reduction tests to ensure hinting and
reporting can reduce the RSS of the guest.

Move the inflation test to performance to ensure it runs sequentially in
CI otherwise the host can be exhausted of huge pages.

Signed-off-by: Jack Thomson <[email protected]>

Update docs to reflect that now huge pages can be used alongside the
traditional balloon device. However, note the caveat that memory may not
be reclaimed from the guest.

Signed-off-by: Jack Thomson <[email protected]>